Personalized screencaps for trickplay slider

ABSTRACT

Systems and methods are provided for displaying personalized preview images during trick-play operations of a media asset based on user profile data and an importance of frames of the media asset. A request from a user to move a slider of a progress bar associated with a media asset is received while the media asset is being generated for display. A scene from the media asset that contains a play position indicated by the slider is identified and a frame that best matches user profile data of the user and importance data for the scene is identified. The identified frame is generated for display as an overlay over the displayed media asset.

BACKGROUND

The present disclosure relates to systems and methods for displayingpreview images during trick-play operations, and, more particularly, fordisplaying personalized preview images during trick-play operations of amedia asset based on user profile data and the importance of frames ofthe media asset.

SUMMARY

Most media players (e.g., including media player applications, mediaplatforms, etc.) provide a variety of trick-play operations to enableusers to navigate through media assets. For example, users canfast-forward, rewind, skip through segments, jump to specific chapters,or perform a “scrubbing” interaction when consuming a media asset.Scrubbing is an interaction in which a user drags their finger (e.g., inthe case of smartphone devices or the like) or a cursor (e.g., in thecase of PCSs or the like) across a segment of the media asset through(e.g., a media player). Media players commonly include a horizontalslider (e.g., a trick-play slider) displayed on a progress bar to enablea user to change the current play position of the media asset beinggenerated for display. In some cases, while the user is dragging theslider, a media player may present previews (e.g., thumbnails) of themedia content corresponding to the position of the slider.

Currently, well-defined requirements for displaying previews duringtrick-play operations are available, e.g., HTTP Live Streaming (HLS),Dynamic Adaptive Streaming over HTTP (DASH), etc. For example, HLSrequires content authors to use I-frame playlists for scrubbing. Also,the use of dedicated I-frame renditions such as one frame/second denseI-frame renditions may be recommended, as opposed to using the I-framesfrom the actual content item, as higher density provides betterperformance for trick mode. Furthermore, the use of multiple I-frameplaylists with different bitrate variants may also be implemented inconventional systems. When a user performs a trick-play operation (e.g.,scrubbing), I-frame renditions are displayed based on the position ofthe slider, the interval of the I-frame renditions, and the speed of thetrick-play operation.

However, in this approach, because previews for an I-frame playlist arespaced at predetermined intervals (e.g., one preview/second, onepreview/three seconds, one preview/minute) and are not personalized fora particular user or based on the importance of the frame, it may bedifficult for a user to quickly navigate to a desired portion of a mediaasset using a trick-play operation (e.g., scrubbing). Additionally,although dense I-frame renditions (e.g., one frame/second) may providemore previews to a user during a trick-play operation, the larger numberof previews may actually make it more difficult for the user to find adesired portion of the media asset.

Accordingly, to solve these problems, systems and methods are providedfor generating a personalized set of previews from select frames of amedia asset (e.g., screencaps) based on a profile of the user and theimportance of the frames to an overall plot to the media asset. Forexample, in some cases, only a single frame may be selected from eachscene of the media asset. Accordingly, when a user performs a scrubbingoperation (e.g., or other trick-play), personalized, high resolution,low-density previews (e.g., one preview/scene) may be shown to the user,so that the user to quickly navigate to a desired time point in themedia asset. In particular, a media player application receives, while amedia asset is being generated for display, a request from a user tomove a slider of a progress bar associated with the media asset. Themedia player application identifies a scene from the media asset thatcontains a play position indicated by the slider, analyzes the scene toidentify a frame that best matches user profile data of the user andimportance data for the scene, and generating for display the identifiedframe overlaid over the displayed media asset. The media playerapplication may repeat the process to display multiple preview imagesduring a trick-play operation.

In some embodiments, the request from the user to move the slider of theprogress bar associated with the media asset may be a scrubbing requestthat changes a play position of the displayed media asset.

In some embodiments, in response to a request to play the media assetfrom the play position indicated by the slider, the media playerapplication may generate for display the media asset from the beginningof the scene.

In some embodiments, the slider may be a virtual slider, and the requestfrom the user to move the slider of the progress bar associated with themedia asset may be a preview scrubbing request that moves the virtualslider without changing a play position of the displayed media asset.

In some embodiments, the media player application, when displaying theidentified frame overlaid over the displayed media asset, may generate ascreencap of the identified frame, the screencap having a lowerresolution than the identified frame, and generate for display thegenerated screencap overlaid over the displayed media asset at the playposition indicated by the slider.

In some embodiments, the media player application, when analyzing thescene to identify the frame, may decode each of a plurality of frames inthe scene, the plurality of frames in the scene may include at least oneI-frame and at least one P-frame, may compare each of the decodedplurality of frames to the user profile data of the user and theimportance data for the scene, and may select, based on a result of thecomparing, one of the decoded plurality of frames as the identifiedframe.

In some embodiments, the media player application, when comparing eachof the decoded plurality of frames to the user profile data of the userand the importance data for the scene, may perform image analysis toeach of the decoded plurality of frames to identify at least one elementin each of the decoded plurality of frames, assign an importance scoreto each of the decoded plurality of frames based on a result ofcomparing the identified at least one element in each of the decodedplurality of frames to the importance data for the scene, assign a userpreference score to each of the decoded plurality of frames based on aresult of comparing the identified at least one element in each of thedecoded plurality of frames to the user profile data of the user, andmay select one of the decoded plurality of frames as the identifiedframe based on the assigned importance score and the assigned userpreference score.

In some embodiments, the importance data for the scene may includemetadata stored in association with the media asset, and the userprofile data of the user may include an actor of interest to the user.

In some embodiments, the identified scene may be a first scene and theidentified frame may be a first frame. In some embodiments, the mediaplayer application may identify a second scene from the media asset thatdoes not contain the play position indicated by the slider, may analyzethe second scene to identify a second frame that best matches userprofile data of the user and importance data for the scene, and generatefor display, while the first frame is being displayed, the identifiedsecond frame overlaid over the displayed media asset.

In some embodiments, the media player application may select theidentified frame as an identifier for the media asset.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and advantages of the disclosure will beapparent upon consideration of the following detailed description, takenin conjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIGS. 1A and 1B illustrate example media asset scrubbing operationdisplay screens, in accordance with some embodiments of the presentdisclosure;

FIG. 2 illustrates an example diagram of a process for selecting a framefrom a scene of a media asset to display as a preview image, inaccordance with some embodiments of the present disclosure;

FIG. 3 illustrates an example media asset scrubbing operation displayscreen for simultaneously displaying multiple preview images while askip scrubbing operation is performed, in accordance with someembodiments of the present disclosure;

FIG. 4 illustrates an example media asset play display screen forresuming playback of a media asset after a scrubbing operation, inaccordance with some embodiments of the present disclosure;

FIGS. 5A and 5B illustrate example media asset recommendation displayscreens for recommending media assets, in accordance with someembodiments of the present disclosure;

FIGS. 6-7 describe exemplary devices, systems, servers, and relatedhardware for

generating a personalized set of previews from select frames of a mediaasset and selectively displaying previews during a trick-play operation,in accordance with some embodiments of the present disclosure;

FIG. 8 depicts a flowchart of illustrative steps of a process forgenerating a preview image of a frame of a media asset selected based ona profile of the user and an importance of the frames to an overall plotto the media asset and displaying the preview image during a trick-playoperation, in accordance with some embodiments of the presentdisclosure; and

FIG. 9 depicts a flowchart of illustrative steps of a process foridentifying and displaying a frame that best matches the user profile asa preview image, among frames of the scene, in accordance with someembodiments of the present disclosure.

DETAILED DESCRIPTION

FIGS. 1A and 1B illustrate example media asset scrubbing operationdisplay screens, in accordance with some embodiments of the presentdisclosure. For example, FIG. 1A illustrates an example media assetscrubbing operation display screen where a preview scrubbing operationis performed, while FIG. 1B illustrates an example media asset scrubbingoperation display screen where a skip scrubbing operation is performed.

In FIG. 1A, a media asset scrubbing operation display screen 104 may bedisplayed on user equipment 102 (e.g., a TV, smartphone, laptop, etc.).The user equipment 102 may comprise control circuitry (described infurther detail with respect to at least FIGS. 6 and 7 ), which executesa media player application to display the media asset scrubbingoperation display screen 104. As shown, the media asset scrubbingoperation display screen 104 displays a media asset 106 (e.g., CaptainAmerica: Civil War), as well as a progress bar 108 and traditionaluser-selectable buttons and indicators to enable a user to navigatethrough the media asset 106. For example, as shown, the progress bar 108includes a slider 110 (e.g., a trick play slider) indicating the currentplay position of the media asset 106. Indicator 112 may indicate00:47:01/01:43:13 as the position of the slider 110 in the progress bar(e.g., indicating the time watched/time remaining of the media asset106). A user may play the media asset 106 (shown as currently paused) byselecting play indicator 114. A user may rewind the media asset 106 byselecting rewind indicator 116 or fast-forward the media asset byselecting fast forward indicator 118. A user may also skip backward orforward a predetermined amount of time (e.g., 10 seconds) by selectingrespective skip indicators 120 and 122.

As shown, the user may perform a preview scrubbing operation by hoveringa cursor 124 over a desired location in the progress bar 108 (e.g.,without moving the slider 110 as described below in FIG. 1B). Inresponse to hovering the cursor 124 over the progress bar 108 as shown(e.g., at 01:29:00 indicated by indicator 126), preview image 128 may bedisplayed overlaid over the media asset 106 (e.g., a portion of themedia asset 106). In contrast to conventional methods of displayingI-frames from I-frame playlists during scrubbing, the preview image 128may be a screencap of a frame in a scene (of the media asset 106) thatcontains a play position indicated by the cursor 124, and may beselected based on an importance of the frame to an overall plot to themedia asset (e.g., compared to other frames in the scene) and userprofile data of the user, as explained in further detail below in FIG. 2. In some embodiments, a scene indicator 130 that corresponds to thescene (e.g., scene 201 of FIG. 2 ) that the cursor is hovering over maybe displayed on the progress bar 108.

In FIG. 1B, a media asset scrubbing operation display screen 132 may bedisplayed on the user equipment 102. The media asset scrubbing operationdisplay screen 132 may correspond to the media asset scrubbing operationdisplay screen 104 described above in FIG. 1A, except that the user mayperform a skip scrubbing operation by selecting the slider 110 with thecursor 124 and dragging the slider 110 to a desired location of theprogress bar 108 (e.g., to the same location as described above in FIG.1A). As shown, during the skip scrubbing operation, the indicator 112may be updated to indicate 01:29:00/01:01:14 (e.g., the changed positionof the slider 110 in the progress bar 108). In response to the userdragging the slider 110 as shown, the preview image 128 may be displayedoverlaid over the media asset 106, as described above in FIG. 1A. Themedia asset 106 may continue to play from the original play position(e.g., 00:47:01/01:43:13) until the user releases the slider 110, atwhich point the media asset 106 may play from the updated play position.In some embodiments, as described in further detail in FIG. 4 , themedia asset may play from the beginning of the scene that the slider 110is released in (e.g., the play position of the slider 110). The user maypause the media asset 106 by selecting pause indicator 134.

Although only a single preview image (e.g., the preview image 128) isshown in the scrubbing operations described above, it should beunderstood that other preview images may be displayed during thescrubbing operations or any other trick-play operation. For example, onepreview image may be displayed for each scene passed during thescrubbing operation. In some embodiments, if the media playerapplication determines that the user will likely not enjoy any of thecontent in a scene of a media asset, the media player application mayrefrain from displaying a preview image for that scene, as described infurther detail in FIG. 2 . In some embodiments, it may be adventitiousto display multiple preview images (corresponding to different scenes)at the same time. For example, a preview image may be displayed for boththe scene where the slider 110 is currently position, as well as asubsequent scene that has not yet been subject to the trick-playoperation, as described in further detail in FIG. 3 .

FIG. 2 illustrates an example diagram 200 of a process for selecting aframe from a scene of a media asset to display as a preview image, inaccordance with some embodiments of the present disclosure. For example,as shown, the diagram 200 shows a process of selecting a frame 214 fromthe scene 201 that is used for generating the preview image 128 shown inFIGS. 1A and 1B. Although only a selection of a single preview for asingle scene is shown, it should be understood that the processdescribed herein may be used to select a frame from each of theplurality of scenes of a media asset (e.g., media asset 106).

In some embodiments, in response to receiving the media asset 106 (e.g.,a stream of the media asset 106), the media player application maydecode the frames of the media asset 106. For example, the media playerapplication may decode each group of pictures (GOPs) including anI-frame (intra-coded frame) and corresponding P-frames (predictiveframes), as well as any B-frames (bi-directional frames). In someembodiments, the media player application may identify the scenes of themedia asset 106 (e.g., by identifying scene changes in metadataassociated with the media asset 106, by leveraging the scene changedetect utilized by the encoder of the media asset 106, or by performingimage analysis to determine a change in characters, locations, orgenre). In some embodiments, the media player application may utilizethe I-frame playlist for analyzing the media asset 106, identifyingasset playback start and decoding points, scene changes, etc. Forexample, in the embodiments shown in FIG. 2 , the media playerapplication may identify the scene 201 (e.g., “Airport Fight Scene(70/110)) having a start time 202 at 1:29:56 and an end time 204 at1:30:50 (e.g., a length of 01:54).

In some embodiments, the media player application may perform imageanalysis and audio analysis of each of the decoded frames to identify atleast one element of interest in each of the decoded frames (e.g., usingimage analysis functions such as facial recognition, object recognition,edge detection, and character recognition of individual frames of videodata, and object tracking across multiple frames of video data, andaudio analysis functions such as waveform analysis of audio data).Because each of the frames is decoded, non-I-frames (e.g., P-frames andB-frames) may be identified as important frames. For example, the mediaplayer application may perform image analysis and audio analysis toidentify certain characters and events in certain frames that correspondto characters or events that are important to a storyline or an overallplot to the media asset 106 (e.g., by comparison to a database of knowncharacters and events), and/or likely to be of interest to a user. Forexample, the media player application may retrieve a user profile 226 ofa user (e.g., the user consuming the media asset 106). The user profilemay indicate, e.g., various interests of the user, viewing history ofthe user, prior search queries of the user, prior interactions withmedia assets by the user, social media interactions by the user relatedto media assets, etc. In one example, as shown, the user profile 226 mayindicate that the user likes “Spiderman,” “Fight scenes,” and“comedies,” but dislikes “romance.” The media player application mayalso retrieve metadata associated with the media asset 106. The metadatamay include information about the media asset 106, such as thedescription, actors, characters, one or more storylines which arenarratives that, along with other narratives, make up the plot of themedia asset 106, etc. For example, the metadata may indicate that“Captain America” played by Chris Evens, “Ironman” played by RobertDowney Jr., and “Spiderman” played by Tom Holland star in the CaptainAmerica: Civil War (i.e., the media asset 106), and that one of thestorylines involves a fight between these characters. The metadata mayalso include scene-specific metadata, which includes informationindicating important characters, events, etc., in each scene and, e.g.,associated time points.

Based on the retrieved metadata and the image analysis of the scene 201,the media player application may identify six frames of interest 206,208, 210, 214, 216, and metadata 218, 220, 222, and 224 associated withsome of the frames. For example, as shown, the media player applicationmay identify the frame 208 as the frame corresponding to the playposition of the slider 110 (e.g., as discussed above in FIGS. 1A and1B). The media player application may identify the frame 206 as thefirst frame in the scene 201. The media player application may identifythe frame 210 when “Ironman removes his helmet,” the frame 212 showing“Captain America before his shield is taken,” the frame 214 when“Spiderman takes Captain America’s shield,” and frame 216 when“Spiderman leaves.” In some embodiments, the media player applicationmay rank the identified frames in a scene based on the relativeimportance of these frames with respect to a plot of the media asset106. For example, the media player application may rank the frame 214 asthe most important frame in the scene 201, followed by the frame 210,the frame 212, and the frame 216. In some embodiments, the media playerapplication may assign a rank value to each of the identified framesbased on their relative importance in the scene 201 as well as the mediaasset 106 as a whole.

The media asset may select one of the identified frames 206, 208, 210,214, 216 by comparing the content of each frame to the user profile 226.For example, because the user profile 226 indicates that the user likesSpiderman, the media player application may select the highest-rankedframe depicting Spiderman (e.g., the frame 214). For another user whodoes not like Spiderman, the media player application may select thehighest-ranked frame not depicting Spiderman (e.g., the frame 210). Foryet another user who loves goodbyes, the media player application mayselect the frame depicting a goodbye (e.g., the frame 216). Thus, inthis example, the media player application may select a frame that isboth important to a plot of the media asset and interesting to the user.

In some embodiments, after selecting the frame 214 for the scene 201,the media player application may generate a preview image (e.g., thepreview image 128 shown in FIGS. 1A and 1B) by generating a screencap ofthe frame 214. In some embodiments, the generated preview image may be ahigh-resolution image (e.g., compared to the I-frame renditions in theI-frame playlist). In some embodiments, the media player application mayinclude a text overlay of the metadata 222 in the generated previewimage 128 (e.g., “Spiderman takes Captain America’s shield”) so that theuser may easily identify a point of interest during a trick-playoperation.

In some embodiments, the media player application may save eachgenerated preview image (e.g., for each scene of the media asset 106) toa personalized preview playlist for the user. In some embodiments, thepersonalized preview playlist may be aligned with the I-frame onlyplaylist such that the selection of a particular preview image allowsthe media player application to start decoding/displaying content from aspecific point in the content item using the I-frame-only playlist, asexplained in further detail below in FIG. 4 . In some embodiments, thepersonalized preview playlist may be generated offline (e.g., before theuser views the media asset 106) or in real-time after the user beginswatching the media asset 106 or performs a trick-play operation. In someembodiments, if the media player application determines that a scene ofthe media asset 106 is not of interest to the user (e.g., a romancescene based on the user profile 226), the media player application mayrefrain from generating a preview image for that scene.

In some embodiments, the media player application may generate atrick-play profile for the user, based on the monitored behavior of theuser (e.g., trick-play operations) and information in the user profileof the user, and generate preview images based on the trick-playprofile. For example, in some embodiments, the trick-play profile of theuser include may include metadata about the user’s dislikes, metadataassociated with previous skips, metadata associated with the user’sinterests in genres such as action/comedy, preferences foractors/actresses, replays, etc. Such metadata can be used to generatethe personalized preview images that predict where the user is likely tofast-forward to during playback. For example, based on previous actionsof the user, if the user previously fast-forwarded through contentdepicting blood (e.g., while watching a documentary) and resumedplayback at the next available scene where blood is not shown, then themedia player application may select a representative frame from a scenethat follows a scene depicting blood for inclusion into the personalizedpreview playlist. In one embodiment, the metadata about the scenesskipped or fast-forwarded is generated offline by means such as videoanalysis, speech analysis, and known image processing techniques inorder to extract metadata and associate such metadata with thetrick-play profile of the user. Similarly, the media player applicationmay rely on structured databases that host granular metadata about videocontent. As one example, the Video Genome Project (VGP) containsdetailed metadata about video items and their relationship to otheritems. Video Genome Project is a trademark owned by Structured DataIntelligence LLC. Such metadata goes beyond traditionalgenre/actor/actress metadata and classifies content in specific nichesin order to bolster content recommendations. Hence the use of deep-scenemetadata may be used to create the personalized preview playlist. Insome embodiments, the media player application may also utilizestructured databases in the identification of important charactersand/or events in certain frames (e.g., during the image and audioanalysis of decoded frames discussed above). In some embodiments, thetrick-play profile for the user may be stored in or in association withthe user profile of the user. In some embodiments, because only alimited number of important preview images may be displayed to a userduring a scrubbing operation, it may be advantageous to display morethan one preview image at a time, as discussed in further detail belowin FIG. 3 .

FIG. 3 illustrates an example media asset scrubbing operation displayscreen 302 for simultaneously displaying multiple preview images while askip scrubbing operation is performed, in accordance with someembodiments of the present disclosure. In FIG. 3 , the media assetscrubbing operation display screen 302 corresponds to the media assetscrubbing operation display screen 132 of FIG. 1B, except that the mediaasset scrubbing operation display screen 302 displays an additionalpreview image 304 (e.g., in addition to the preview image 128 displayedbased on the current location of the slider 110/cursor 124 during thescrubbing operation) at a subsequent play position 306 that does notcorrespond to the current location of the slider 110/cursor 124. Thus,the user may more quickly identify a point of interest in the mediaasset 106.

FIG. 4 illustrates an example media asset play display screen 402 forresuming playback of a media asset after a scrubbing operation, inaccordance with some embodiments of the present disclosure. In responseto the user stopping the trick-play operation described in FIG. 1A(e.g., releasing the slider 110 at the play position of 01:29:00), themedia player application may resume playback of the media asset 106 atthe corresponding play position indicated by the trick-play operation(01:29:00). However, in some embodiments, it may be advantageous toidentify the beginning of the scene indicated by the slider 110 andresume playback from the beginning of the scene. For example, as shownin the example media asset play display screen 402, in response to theuser stopping the slider 110 at play position 01:29:00, the media playerapplication may resume playback at the beginning of the scene (e.g., at01:28:56 indicated by the updated indicator 112). To do this, the mediaplayer application may, in some embodiments, determine the I-frame inthe I-frame playlist that is mapped to the displayed preview image(e.g., an earlier I-frame at the beginning of the scene). Thus, in someembodiments, because the user does not have to perform anothertrick-play operation to find the beginning of the scene, processing bythe media player application may be reduced, while improving userexperience. In some embodiments, for scenes longer than a threshold time(e.g., 3 minutes), it may be advantageous to resume playback at thelocation of the displayed preview image (e.g., 01:29:35 for the previewimage). In some embodiments, the media player application may monitorthe behavior of the user after playback is resumed and adjust playbackthresholds (e.g., if the user always performs an additional trick-playoperation to find the frame included in the preview image, the mediaplayer application may begin playback from this point).

FIGS. 5A and 5B illustrate example media asset recommendation displayscreens for recommending media assets, in accordance with someembodiments of the present disclosure. For example, FIG. 5A illustratesan initial media asset recommendation display screen 502 displayed onthe user equipment 102. As shown, the initial media asset recommendationdisplay screen 502 may display a first indicator 506 for a first mediaasset recommendation 504 (“Captain America: Civil War) and a secondindicator 510 for a second media asset recommendation 508 (“Spiderman”).If, for example, the user consumes the second media asset recommendation508 (“Spiderman”), the media player application may update the userprofile of the user, and display the updated media asset recommendationdisplay screen 512 shown in FIG. 5B.

In FIG. 5B, the updated media asset recommendation display screen 512may provide a new indicator 516 for a new media asset recommendation 514(“Spiderman: Far From Home”). Additionally, the media player applicationmay continue to recommend the first media asset recommendation 504(“Captain America: Civil War), but with an updated indicator 518. Forexample, based on a determination that the user is interested inSpiderman, the media player application may select one of the previewimages generated for the user for Captain America: Civil War (e.g., thepreview image 128 depicting Spiderman) as the indicator for the mediaasset recommendation 504. Additionally, media asset identifiers that auser has started watching (e.g., displayed on a continue watching page)may be replaced with the personalized preview image closest to the pointwhere the user stopped watching the media asset. However, these are onlyexamples, and the personalized preview images may be used in anysuitable manner to personalize the experience of the user.

FIGS. 6-7 describe exemplary devices, systems, servers, and relatedhardware for generating a personalized set of previews from selectframes of a media asset and selectively displaying previews during atrick-play operation, in accordance with some embodiments of the presentdisclosure. FIG. 6 shows generalized embodiments of illustrative userequipment devices 600 and 601. For example, user equipment device 600may be a tablet or smartphone device. In another example, user equipmentsystem 601 may be a user television equipment system (e.g., the userequipment 102). In another example, the user equipment system 601 may bea stereo or other audio playback device. The user television equipmentsystem 601 may include a set-top box 616. The set-top box 616 may becommunicatively connected to a camera 618, a speaker 614, and a display612. In some embodiments, the camera may detect users viewing thedisplay. In some embodiments, the display 612 may be a televisiondisplay or a computer display. In some embodiments, the set-top box 616may be communicatively connected to a user input interface 610. In someembodiments, the user input interface 610 may be a remote controldevice. The set-top box 616 may include one or more circuit boards. Insome embodiments, the circuit boards may include processing circuitry,control circuitry, and storage (e.g., RAM, ROM, hard disk, removabledisk, etc.). In some embodiments, the circuit boards may include aninput/output path. More specific implementations of user equipmentdevices are discussed below in connection with FIG. 7 . Each one of theuser equipment device 600 and the user equipment system 601 may receivecontent and data via input/output (I/O) path 602. The I/O path 602 mayprovide content (e.g., broadcast programming, on-demand programming,Internet content, content available over a local area network (LAN) orwide area network (WAN), and/or other content) and data to controlcircuitry 604, which includes processing circuitry 606 and a storage608. The control circuitry 604 may be used to send and receive commands,requests, and other suitable data using the I/O path 602. The I/O path602 may connect the control circuitry 604 (and specifically theprocessing circuitry 606) to one or more communications paths (describedbelow). I/O functions may be provided by one or more of thesecommunications paths, but are shown as a single path in FIG. 6 to avoidovercomplicating the drawing.

The control circuitry 604 may be based on any suitable processingcircuitry such as the processing circuitry 606. As referred to herein,processing circuitry should be understood to mean circuitry based on oneor more microprocessors, microcontrollers, digital signal processors,programmable logic devices, field-programmable gate arrays (FPGAs),application-specific integrated circuits (ASICs), etc., and may includea multi-core processor (e.g., dual-core, quad-core, hexa-core, or anysuitable number of cores) or supercomputer. In some embodiments,processing circuitry may be distributed across multiple separateprocessors or processing units, for example, multiple of the same typeof processing units (e.g., two Intel Core i7 processors) or multipledifferent processors (e.g., an Intel Core i5 processor and an Intel Corei7 processor). In some embodiments, the control circuitry 604 executesinstructions for a media player application stored in memory (i.e., thestorage 608). Specifically, the control circuitry 604 may be instructedby the media player application to perform the functions discussed aboveand below. In some implementations, any action performed by the controlcircuitry 604 may be based on instructions received from the mediaplayer application.

In client/server-based embodiments, the control circuitry 604 mayinclude communications circuitry suitable for communicating with a mediaplayer application server or other networks or servers. The instructionsfor carrying out the above-mentioned functionality may be stored on aserver (which is described in more detail in connection with FIG. 7 ).Communications circuitry may include a cable modem, an integratedservices digital network (ISDN) modem, a digital subscriber line (DSL)modem, a telephone modem, an Ethernet card, or a wireless modem forcommunications with other equipment, or any other suitablecommunications circuitry. Such communications may involve the Internetor any other suitable communication networks or paths (which isdescribed in more detail in connection with FIG. 7 ). In addition,communications circuitry may include circuitry that enables peer-to-peercommunication of user equipment devices, or communication of userequipment devices in locations remote from each other (described in moredetail below).

Memory may be an electronic storage device provided as the storage 608that is part of the control circuitry 604. As referred to herein, thephrase “electronic storage device” or “storage device” should beunderstood to mean any device for storing electronic data, computersoftware, or firmware, such as random-access memory, read-only memory,hard drives, optical drives, digital video recorders (DVRs, sometimescalled personal video recorders, or PVRs), solid state devices, quantumstorage devices, gaming consoles, gaming media, or any other suitablefixed or removable storage devices, and/or any combination of the same.The storage 608 may be used to store various types of content describedherein as well as media player application data described above. Forexample, the storage 608 may be used to store the user profile describedherein Nonvolatile memory may also be used (e.g., to launch a boot-uproutine and other instructions). Cloud-based storage, described inrelation to FIG. 7 , may be used to supplement the storage 608 orinstead of the storage 608.

The control circuitry 604 may include video generating circuitry andtuning circuitry, such as one or more analog tuners, one or more MPEG-2decoders or other digital decoding circuitry, high-definition tuners, orany other suitable tuning or video circuits or combinations of suchcircuits. Encoding circuitry (e.g., for converting over-the-air, analog,or digital signals to MPEG signals for storage) may also be provided.The control circuitry 604 may also include scaler circuitry forupconverting and downconverting content into the preferred output formatof the user equipment 600. The circuitry 604 may also includedigital-to-analog converter circuitry and analog-to-digital convertercircuitry for converting between digital and analog signals. The tuningand encoding circuitry may be used by the user equipment device toreceive and to display, to play, or to record content. The tuning andencoding circuitry may also be used to receive guidance data. Thecircuitry described herein, including for example, the tuning, videogenerating, encoding, decoding, encrypting, decrypting, scaler, andanalog/digital circuitry, may be implemented using software running onone or more general purpose or specialized processors. Multiple tunersmay be provided to handle simultaneous tuning functions (e.g., watch andrecord functions, picture-in-picture (PIP) functions, multiple-tunerrecording, etc.). If the storage 608 is provided as a separate devicefrom the user equipment device 600, the tuning and encoding circuitry(including multiple tuners) may be associated with the storage 608.

A user may send instructions to the control circuitry 604 using the userinput interface 610. The user input interface 610 may be any suitableuser interface, such as a remote control, mouse, trackball, keypad,keyboard, touchscreen, touchpad, stylus input, joystick, voicerecognition interface, or other user input interfaces. The display 612may be provided as a stand-alone device or integrated with otherelements of each one of the user equipment device 600 and the userequipment system 601. For example, the display 612 may be a touchscreenor touch-sensitive display. In such circumstances, the user inputinterface 610 may be integrated with or combined with display 612. Thedisplay 612 may be one or more of a monitor, a television, a display fora mobile device, or any other type of display. A video card or graphicscard may generate the output to the display 612. The video card may beany processing circuitry described above in relation to the controlcircuitry 604. The video card may be integrated with the controlcircuitry 604. Speakers 614 may be provided as integrated with otherelements of each one of the user equipment device 600 and the userequipment system 601 or may be stand-alone units. The audio component ofvideos and other content displayed on the display 612 may be playedthrough the speakers 614. In some embodiments, the audio may bedistributed to a receiver (not shown), which processes and outputs theaudio via speakers 614.

The media player application may be implemented using any suitablearchitecture. For example, it may be a stand-alone application whollyimplemented on each one of the user equipment device 600 and the userequipment system 601. In such an approach, instructions of theapplication are stored locally (e.g., in the storage 608), and data foruse by the application is downloaded on a periodic basis (e.g., from anout-of-band feed, from an Internet resource, or using another suitableapproach). The control circuitry 604 may retrieve instructions of theapplication from the storage 608 and process the instructions torecommend media content items as discussed. Based on the processedinstructions, the control circuitry 604 may determine what action toperform when input is received from the user input interface 610. Forexample, movement of a cursor on a display up/down may be indicated bythe processed instructions when the user input interface 610 indicatesthat an up/down button was selected.

In some embodiments, the media player application is aclient/server-based application. Data for use by a thick or thin clientimplemented on each one of the user equipment device 600 and the userequipment system 601 is retrieved on demand by issuing requests to aserver remote to each one of the user equipment device 600 and the userequipment system 601. In one example of a client/server-based guidanceapplication, the control circuitry 604 runs a web browser thatinterprets web pages provided by a remote server. For example, theremote server may store the instructions for the application in astorage device. The remote server may process the stored instructionsusing circuitry (e.g., the control circuitry 604) and generate apersonalized set of previews from select frames of a media asset andselectively display previews during trick-play operations as discussed.

In some embodiments, the media player application is downloaded andinterpreted or otherwise run by an interpreter or virtual machine (runby the control circuitry 604). In some embodiments, the media playerapplication may be encoded in the ETV Binary Interchange Format (EBIF),received by the control circuitry 604 as part of a suitable feed, andinterpreted by a user agent running on the control circuitry 604. Forexample, the media player application may be an EBIF application. Insome embodiments, the media player application may be defined by aseries of JAVA-based files that are received and run by a local virtualmachine or other suitable middleware executed by control circuitry 604.In some of such embodiments (e.g., those employing MPEG-2 or otherdigital media encoding schemes), the media player application may be,for example, encoded and transmitted in an MPEG-2 object carousel withthe MPEG audio and video packets of a program.

FIG. 7 is a diagram of an illustrative media system, in accordance withsome embodiments of the disclosure. User equipment devices 708 and 710(such as the user equipment 102) may be coupled to communication network706. The communication network 706 may be one or more networks includingthe Internet, a mobile phone network, mobile voice or data network(e.g., a 4G or LTE network), cable network, public switched telephonenetwork, or other types of communication network or combinations ofcommunication networks. Paths (e.g., depicted as arrows connecting therespective devices to the communication network 706) may separately ortogether include one or more communications paths, such as a satellitepath, a fiber-optic path, a cable path, a path that supports Internetcommunications (e.g., IPTV), free-space connections (e.g., for broadcastor other wireless signals), or any other suitable wired or wirelesscommunications path or combination of such paths. Communications withthe client devices may be provided by one or more of thesecommunications paths but are shown as a single path in FIG. 7 to avoidovercomplicating the drawing.

Although communications paths are not drawn between user equipmentdevices, these devices may communicate directly with each other viacommunications paths such as short-range communication paths,point-to-point communications paths, such as USB cables, IEEE 1394cables, wireless paths (e.g., Bluetooth, infrared, IEEE 802-11x, etc.),or other short-range communication via wired or wireless paths. The userequipment devices may also communicate with each other directly throughan indirect path via the communication network 706.

The system 700 includes a media content source 702 and a server 704.Communications with the media content source 702 and the server 704 maybe exchanged over one or more communications paths but are shown as asingle path in FIG. 7 to avoid overcomplicating the drawing. Inaddition, there may be more than one of each of the media content source702 and the server 704, but only one of each is shown in FIG. 7 to avoidovercomplicating the drawing. If desired, the media content source 702and the server 704 may be integrated as one source device.

In some embodiments, the server 704 may include control circuitry 711and a storage 714 (e.g., RAM, ROM, hard disk, removable disk, etc.). Theserver 704 may also include an input/output path 712. The I/O path 712may provide device information, or other data, over a local area network(LAN) or wide area network (WAN), and/or other content and data to thecontrol circuitry 711, which includes processing circuitry, and thestorage 714. The control circuitry 711 may be used to send and receivecommands, requests, and other suitable data using the I/O path 712. TheI/O path 712 may connect the control circuitry 711 (and specificallyprocessing circuitry) to one or more communications paths.

The control circuitry 711 may be based on any suitable processingcircuitry such as one or more microprocessors, microcontrollers, digitalsignal processors, programmable logic devices, field-programmable gatearrays (FPGAs), application-specific integrated circuits (ASICs), etc.,and may include a multi-core processor (e.g., dual-core, quad-core,hexa-core, or any suitable number of cores) or supercomputer. In someembodiments, control circuitry 711 may be distributed across multipleseparate processors or processing units, for example, multiple of thesame type of processing units (e.g., two Intel Core i7 processors) ormultiple different processors (e.g., an Intel Core i5 processor and anIntel Core i7 processor). In some embodiments, the control circuitry 711executes instructions for an emulation system application stored inmemory (e.g., the storage 714). Memory may be an electronic storagedevice provided as the storage 714 that is part of the control circuitry711.

The server 704 may retrieve guidance data from media content source 702,process the data as will be described in detail below, and forward thedata to the user equipment devices 708 and 710. The media content source702 may include one or more types of content distribution equipmentincluding an audio distribution facility, a television distributionfacility, cable system headend, satellite distribution facility,programming sources (e.g., television broadcasters, such as NBC, ABC,HBO, etc.), intermediate distribution facilities and/or servers,Internet providers, on-demand media servers, and other contentproviders. NBC is a trademark owned by the National BroadcastingCompany, Inc., ABC is a trademark owned by the American BroadcastingCompany, Inc., and HBO is a trademark owned by the Home Box Office, Inc.Media content source 702 may be the originator of content (e.g., atelevision broadcaster, a Webcast provider, etc.) or may not be theoriginator of content (e.g., an on-demand content provider, an Internetprovider of content of broadcast programs for downloading, etc.). Themedia content source 702 may include cable sources, satellite providers,on-demand providers, Internet providers, over-the-top content providers,or other providers of content. The media content source 702 may alsoinclude a remote media server used to store different types of content(including video content selected by a user), in a location remote fromany of the client devices.

Client devices may operate in a cloud computing environment to accesscloud services. In a cloud computing environment, various types ofcomputing services for content sharing, storage or distribution (e.g.,video sharing sites or social networking sites) are provided by acollection of network-accessible computing and storage resources,referred to as “the cloud.” For example, the cloud can include acollection of server computing devices (such as, e.g., server 704),which may be located centrally or at distributed locations, that providecloud-based services to various types of users and devices connected viaa network (such as the Internet) via communication network 706. In suchembodiments, user equipment devices may operate in a peer-to-peer mannerwithout communicating with a central server.

FIG. 8 depicts a flowchart of illustrative steps of a process 800 forgenerating a preview image of a frame of a media asset selected based ona profile of the user and an importance of the frames to an overall plotto the media asset and displaying the preview image during a trick-playoperation, in accordance with some embodiments of the presentdisclosure.

The process 800 may be executed by the control circuitry 604 (e.g., in amanner instructed to the control circuitry 604 by the media playerapplication). The control circuitry 604 may be part of user equipment(e.g., a device that may have any or all of the functionality of theuser equipment devices 708 and 710), or of a remote server separatedfrom the user equipment by way of the communication network 706, ordistributed over a combination of both. It should be noted that theprocess 800, or any step thereof, could be performed on, or provided by,any of the devices described above in FIGS. 1A, 1B, 3, 4, 5A, and 5B.

The process 800 begins at step 802 when the media player application(e.g., via the control circuitry 604) receives a request from a user tomove a slider of a progress bar associated with a displayed media asset.For example, the request may be a request for a trick-play operationsuch as a scrubbing operation.

At 804, the media player application (e.g., via the control circuitry604) identifies the user and retrieves a user profile of the user. Forexample, the control circuitry 604 may identify the user using varioustechniques such as facial recognition techniques, voice recognitiontechniques, a profile currently logged into the user equipment used toview the media asset, information obtained from a device associated withthe user, etc. The user profile of the user may indicate, e.g., variousinterests of the user, viewing history of the user, prior search queriesof the user, prior interactions with media assets by the user, socialmedia interactions by the user related to media assets, etc.

At 806, the media player application (e.g., via the control circuitry604) identifies a scene from the media asset that contains the playposition indicated by the slider. For example, the control circuitry 604may identify the start and end times of scenes in the media assets anddetermine which scene the play position is currently located in (e.g.,during the trick play operation).

At 808, the media player application (e.g., via the control circuitry604) analyzes the scene to identify a frame that best matches the userprofile and importance data for the scene, as explained in furtherdetail in FIG. 9 .

At 810, the media player application (e.g., via the control circuitry604) generates for display the identified frame overlaid over thedisplayed media asset. For example, as explained in further detail inFIG. 9 , the control circuitry may generate for display a thumbnailpreview image at the play position indicated by the slider.

At 812, the media player application (e.g., via the control circuitry604) determines if the play position indicated by the slider has beenmoved to a new scene. For example, the control circuitry 604 hascontinued the trick-play operation (e.g., continued scrubbing). If thecontrol circuitry 604 determines that the play position indicated by theslider has been moved to a new scene (“Yes” at step 812), the process800 returns to step 808 (e.g., to identify and display the next previewimage). Otherwise (“No” at step 812), the process 800 proceeds to step814.

At 814, the media player application (e.g., via the control circuitry604) determines if a request to resume playback of the media asset atthe play position indicated by the slider has been received. Forexample, the control circuitry 604 determines if the user has stoppedthe trick-play to resume playback. If the control circuitry 604determines that a request resume playback of the media asset has beenreceived (“Yes” at step 814), the process 800 proceeds to step 816.Otherwise (“No” at step 814), the process 800 returns to step 812 andmonitors the play position indicated by the slider.

At 816, the media player application (e.g., via the control circuitry604) identifies the beginning of the scene that contains the playposition indicated by the slider.

At 818, the media player application (e.g., via the control circuitry604) resumes playback of the media asset from the beginning of thescene.

FIG. 9 depicts a flowchart of illustrative steps of a process 900 foridentifying and displaying a frame that best matches the user profile asa preview image, among frames of the scene, in accordance with someembodiments of the present disclosure. The process 900 is one embodimentof a method for performing steps 808 and 810 of FIG. 8 and begins afterstep 806. At 902, the media player application (e.g., via the controlcircuitry 604) decodes frames of the identified scene (e.g., includingany P-frames or B-frames).

At 904, the media player application (e.g., via the control circuitry604) retrieves metadata associated with the media asset. The metadatamay include information about the media asset (e.g., including one ormore storylines that make up the plot of the media asset) and scenespecific metadata which includes information indicating importantcharacters, events, etc. in the identified scene.

At 906, the media player application (e.g., via the control circuitry604) analyzes the retrieved metadata to identify importance data for thescene (e.g., indicating the rank and importance of events in the scene).

At 908, the media player application (e.g., via the control circuitry604) analyzes the decoded frames to identify frames matching theimportance data for the scene. For example, the control circuitry 604identifies frames depicting the important events in the scene.

At 910, the media player application (e.g., via the control circuitry604) determines an importance data for each of the identified frames.For example, the control circuity 604 may rank the identified framesfrom most important to least important and/or assign an importance valuerepresenting the importance of the identified frames within both thescene and the media asset.

At 912, the media player application (e.g., via the control circuitry604) selects one of the identified frames based the determinedimportance rank and user interests in the retrieved user profile. Forexample, in one embodiment the control circuitry 604 weights thedetermined importance rank and matching user interests to determine acomposite importance score for each of the identified frames and selectsthe frame with the highest score.

At 914, the media player application (e.g., via the control circuitry604) generates a preview image from the selected frame. For example, thecontrol circuitry 604 may generate a preview image by generating ascreencap of the frame. In some embodiments, the generated preview imagemay have a larger resolution than the I-frame renditions in the I-frameplaylist.

At 916, the media player application (e.g., via the control circuitry604) generates for display the generated preview image overlaid over thedisplayed media asset.

The processes discussed above are intended to be illustrative and notlimiting. One skilled in the art would appreciate that the steps of theprocesses discussed herein may be omitted, modified, combined and/orrearranged, and any additional steps may be performed without departingfrom the scope of the invention. For example, step 816 may be omittedand playback of the media asset may be started from the position of theslider of the location of the displayed preview. More generally, theabove disclosure is meant to be exemplary and not limiting. Only theclaims that follow are meant to set bounds as to what the presentinvention includes. Furthermore, it should be noted that the featuresand limitations described in any one embodiment may be applied to anyother embodiment herein, and flowcharts or examples relating to oneembodiment may be combined with any other embodiment in a suitablemanner, done in different orders, or done in parallel. In addition, thesystems and methods described herein may be performed in real-time. Itshould also be noted that the systems and/or methods described above maybe applied to, or used in accordance with, other systems and/or methods.

1-30. (canceled)
 31. A method comprising: receiving, while a media assetis being generated for display, a request from a user to move a sliderof a progress bar associated with the media asset; identifying a scenefrom the media asset that contains a play position indicated by theslider; analyzing the scene to identify a frame that best matches userprofile data of the user and importance data for the scene, wherein theimportance data indicates the importance of frames of the scene to anoverall plot of the media asset; and generating for display theidentified frame overlaid over the displayed media asset, wherein theanalyzing the scene to identify the frame comprises: decoding each of aplurality of frames in the scene, wherein the plurality of frames in thescene comprises at least one I-frame and at least one P-frame; comparingeach of the decoded plurality of frames to the user profile data of theuser and the importance data for the scene; and selecting, based on aresult of the comparing, one of the decoded plurality of frames as theidentified frame.
 32. The method of claim 31, wherein the request fromthe user to move the slider of the progress bar associated with themedia asset is a scrubbing request, and wherein the scrubbing requestchanges a play position of the displayed media asset.
 33. The method ofclaim 32, further comprising in response to a request to play the mediaasset from the play position indicated by the slider, generating fordisplay the media asset from the beginning of the scene.
 34. The methodof claim 31, wherein the displaying the identified frame overlaid overthe displayed media asset comprises: generating a screencap of theidentified frame, wherein the screencap has a lower resolution than theidentified frame; and generating for display the generated screencapoverlaid over the displayed media asset at the play position indicatedby the slider.
 35. The method of claim 31, wherein the comparing each ofthe decoded plurality of frames to the user profile data of the user andthe importance data for the scene comprises: performing image analysisto each of the decoded plurality of frames to identify at least oneelement in each of the decoded plurality of frames; assigning animportance score to each of the decoded plurality of frames based on aresult of comparing the identified at least one element in each of thedecoded plurality of frames to the importance data for the scene;assigning a user preference score to each of the decoded plurality offrames based on a result of comparing the identified at least oneelement in each of the decoded plurality of frames to the user profiledata of the user; and selecting one of the decoded plurality of framesas the identified frame based on the assigned importance score and theassigned user preference score.
 36. The method of claim 31, wherein theimportance data for the scene comprises metadata stored in associationwith the media asset, and wherein the user profile data of the usercomprises an actor of interest to the user.
 37. The method of claim 31,wherein the identified scene is a first scene and the identified frameis a first frame, and wherein the method further comprises: identifyinga second scene from the media asset that does not contain the playposition indicated by the slider; analyzing the second scene to identifya second frame that best matches user profile data of the user andimportance data for the scene; and generating for display, while thefirst frame is being displayed, the identified second frame overlaidover the displayed media asset.
 38. The method of claim 31, furthercomprising selecting the identified frame as an identifier for the mediaasset.
 39. A system comprising: a memory storing instructions; andcontrol circuitry configured to execute the instructions stored in thememory to: receive, while a media asset is being generated for display,a preview scrubbing request from a user to move a slider of a progressbar associated with the media asset; identify a scene from the mediaasset that contains a play position indicated by the slider; analyze thescene to identify a frame that best matches user profile data of theuser and importance data for the scene, wherein the importance dataindicates the importance of frames of the scene to an overall plot ofthe media asset; and generate for display the identified frame overlaidover the displayed media asset, wherein the control circuitry is furtherconfigured, when analyzing the scene to identify the frame: to decodeeach of a plurality of frames in the scene, wherein the plurality offrames in the scene comprises at least one I-frame and at least oneP-frame; to compare each of the decoded plurality of frames to the userprofile data of the user and the importance data for the scene; and toselect, based on a result of the comparing, one of the decoded pluralityof frames as the identified frame.
 40. The system of claim 39, whereinthe request from the user to move the slider of the progress barassociated with the media asset is a scrubbing request, and wherein thescrubbing request changes a play position of the displayed media asset.41. The system of claim 40, wherein the control circuitry is furtherconfigured, in response to a request to play the media asset from theplay position indicated by the slider, generate for display the mediaasset from the beginning of the scene.
 42. The system of claim 39,wherein the control circuitry is further configured, when displaying theidentified frame overlaid over the displayed media asset, to: generate ascreencap of the identified frame, wherein the screencap has a lowerresolution than the identified frame; and generate for display thegenerated screencap overlaid over the displayed media asset at the playposition indicated by the slider.
 43. The system of claim 39, whereinthe control circuitry is further configured, when comparing each of thedecoded plurality of frames to the user profile data of the user and theimportance data for the scene, to: perform image analysis to each of thedecoded plurality of frames to identify at least one element in each ofthe decoded plurality of frames; assign an importance score to each ofthe decoded plurality of frames based on a result of comparing theidentified at least one element in each of the decoded plurality offrames to the importance data for the scene; assign a user preferencescore to each of the decoded plurality of frames based on a result ofcomparing the identified at least one element in each of the decodedplurality of frames to the user profile data of the user; and select oneof the decoded plurality of frames as the identified frame based on theassigned importance score and the assigned user preference score. 44.The system of claim 39, wherein the importance data for the scenecomprises metadata stored in association with the media asset, andwherein the user profile data of the user comprises an actor of interestto the user.
 45. The system of claim 39, wherein the identified scene isa first scene and the identified frame is a first frame, and wherein thecontrol circuitry is further configured to: identify a second scene fromthe media asset that does not contain the play position indicated by theslider; analyze the second scene to identify a second frame that bestmatches user profile data of the user and importance data for the scene;and generate for display, while the first frame is being displayed, theidentified second frame overlaid over the displayed media asset.
 46. Thesystem of claim 39, wherein the control circuitry is further configuredto select the identified frame as an identifier for the media asset.